{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploy and predict with Keras model on Cloud AI Platform.\n",
    "\n",
    "**Learning Objectives**\n",
    "\n",
    "1. Setup up the environment\n",
    "1. Deploy trained Keras model to Cloud AI Platform\n",
    "1. Online predict from model on Cloud AI Platform\n",
    "1. Batch predict from model on Cloud AI Platform\n",
    "\n",
    "\n",
    "## Introduction \n",
    "**Verify that you have previously Trained your Keras model. If not, go back to [train_keras_ai_platform_babyweight.ipynb](../solutions/train_keras_ai_platform_babyweight.ipynb) create them.**\n",
    "In this notebook, we'll be deploying our Keras model to Cloud AI Platform and creating predictions.\n",
    "\n",
    "We will set up the environment, deploy a trained Keras model to Cloud AI Platform, online predict from deployed model on Cloud AI Platform, and batch predict from deployed model on Cloud AI Platform.\n",
    "\n",
    "Each learning objective will correspond to a __#TODO__ in this student lab notebook -- try to complete this notebook first and then review the [solution notebook](../solutions/deploy_keras_ai_platform_babyweight.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hJ7ByvoXzpVI"
   },
   "source": [
    "## Set up environment variables and load necessary libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import necessary libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lab Task #1: Set environment variables.\n",
    "\n",
    "Set environment variables so that we can use them throughout the entire lab. We will be using our project name for our bucket, so you only need to change your project and region."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "Your current GCP Project Name is: michaelabel-gcp-training\n"
      ]
     }
   ],
   "source": [
    "%%bash\n",
    "PROJECT=$(gcloud config list project --format \"value(core.project)\")\n",
    "echo \"Your current GCP Project Name is: \"$PROJECT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Change these to try this notebook out\n",
    "PROJECT = \"cloud-training-demos\"  # TODO 1: Replace with your PROJECT\n",
    "BUCKET = PROJECT  # defaults to PROJECT\n",
    "REGION = \"us-central1\"  # TODO 1: Replace with your REGION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.environ[\"BUCKET\"] = BUCKET\n",
    "os.environ[\"REGION\"] = REGION\n",
    "os.environ[\"TFVERSION\"] = \"2.1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stderr",
      "output_type": "stream",
      "text": [
       "Updated property [compute/region].\n",
       "Updated property [ai_platform/region]."
      ]
     }
   ],
   "source": [
    "%%bash\n",
    "gcloud config set compute/region $REGION\n",
    "gcloud config set ai_platform/region global"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check our trained model files\n",
    "\n",
    "Let's check the directory structure of our outputs of our trained model in folder we exported the model to in our last [lab](../solutions/10_train_keras_ai_platform_babyweight.ipynb). We'll want to deploy the saved_model.pb within the timestamped directory as well as the variable values in the variables folder. Therefore, we need the path of the timestamped directory so that everything within it can be found by Cloud AI Platform's model deployment service."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "gs://michaelabel-gcp-training-ml/babyweight/trained_model/\n",
       "gs://michaelabel-gcp-training-ml/babyweight/trained_model/20200428154223/\n",
       "gs://michaelabel-gcp-training-ml/babyweight/trained_model/20200428221424/\n",
       "gs://michaelabel-gcp-training-ml/babyweight/trained_model/20200428230409/\n",
       "gs://michaelabel-gcp-training-ml/babyweight/trained_model/checkpoints/\n"
      ]
     }
   ],
   "source": [
    "%%bash\n",
    "gcloud storage ls gs://${BUCKET}/babyweight/trained_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "gs://michaelabel-gcp-training-ml/babyweight/trained_model/20200428230409/\n",
       "gs://michaelabel-gcp-training-ml/babyweight/trained_model/20200428230409/saved_model.pb\n",
       "gs://michaelabel-gcp-training-ml/babyweight/trained_model/20200428230409/assets/\n",
       "gs://michaelabel-gcp-training-ml/babyweight/trained_model/20200428230409/variables/\n"
      ]
     }
   ],
   "source": [
    "%%bash\n",
    "MODEL_LOCATION=$(gcloud storage ls -- gs://${BUCKET}/babyweight/trained_model/2* | grep '/$' \\\n",
    "                 | tail -1)\n",
    "gcloud storage ls ${MODEL_LOCATION}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab Task #2: Deploy trained model.\n",
    "\n",
    "Deploying the trained model to act as a REST web service is a simple gcloud call. Complete __#TODO__ by providing location of saved_model.pb file to Cloud AI Platform model deployment service. The deployment will take a few minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "Deleting and deploying babyweight ml_on_gcp from gs://michaelabel-gcp-training-ml/babyweight/trained_model/20200428230409/\n"
      ]
     },
     {
      "name": "stderr",
      "output_type": "stream",
      "text": [
       "WARNING: Using endpoint [https://ml.googleapis.com/]\n",
       "Created ml engine model [projects/michaelabel-gcp-training/models/babyweight].\n",
       "WARNING: Using endpoint [https://ml.googleapis.com/]\n",
       "Creating version (this might take a few minutes)......\n",
       ".............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................done.\n"
      ]
     }
   ],
   "source": [
    "%%bash\n",
    "MODEL_NAME=\"babyweight\"\n",
    "MODEL_VERSION=\"ml_on_gcp\"\n",
    "MODEL_LOCATION=# TODO 2: Add GCS path to saved_model.pb file.\n",
    "echo \"Deleting and deploying $MODEL_NAME $MODEL_VERSION from $MODEL_LOCATION\"\n",
    "# gcloud ai-platform versions delete ${MODEL_VERSION} --model ${MODEL_NAME}\n",
    "# gcloud ai-platform models delete ${MODEL_NAME}\n",
    "gcloud ai-platform models create ${MODEL_NAME} --regions ${REGION}\n",
    "gcloud ai-platform versions create ${MODEL_VERSION} \\\n",
    "    --model=${MODEL_NAME} \\\n",
    "    --origin=${MODEL_LOCATION} \\\n",
    "    --runtime-version=2.1 \\\n",
    "    --python-version=3.7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab Task #3: Use model to make online prediction.\n",
    "\n",
    "Complete __#TODO__s for both the Python and gcloud Shell API methods of calling our deployed model on Cloud AI Platform for online prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python API\n",
    "\n",
    "We can use the Python API to send a JSON request to the endpoint of the service to make it predict a baby's weight. The order of the responses are the order of the instances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "b'{\"predictions\": [{\"weight\": [6.434544563293457]}, {\"weight\": [6.527022361755371]}, {\"weight\": [3.9038546085357666]}, {\"weight\": [6.076201438903809]}]}'\n"
      ]
     }
   ],
   "source": [
    "from oauth2client.client import GoogleCredentials\n",
    "import requests\n",
    "import json\n",
    "\n",
    "MODEL_NAME = # TODO 3a: Add model name\n",
    "MODEL_VERSION = # TODO 3a: Add model version\n",
    "\n",
    "token = GoogleCredentials.get_application_default().get_access_token().access_token\n",
    "api = \"https://ml.googleapis.com/v1/projects/{}/models/{}/versions/{}:predict\" \\\n",
    "         .format(PROJECT, MODEL_NAME, MODEL_VERSION)\n",
    "headers = {\"Authorization\": \"Bearer \" + token }\n",
    "data = {\n",
    "  \"instances\": [\n",
    "    {\n",
    "      \"is_male\": \"True\",\n",
    "      \"mother_age\": 26.0,\n",
    "      \"plurality\": \"Single(1)\",\n",
    "      \"gestation_weeks\": 39\n",
    "    },\n",
    "    {\n",
    "      \"is_male\": \"False\",\n",
    "      \"mother_age\": 29.0,\n",
    "      \"plurality\": \"Single(1)\",\n",
    "      \"gestation_weeks\": 38\n",
    "    },\n",
    "    {\n",
    "      \"is_male\": \"True\",\n",
    "      \"mother_age\": 26.0,\n",
    "      \"plurality\": \"Triplets(3)\",\n",
    "      \"gestation_weeks\": 39\n",
    "    },\n",
    "    # TODO 3a: Create another instance\n",
    "  ]\n",
    "}\n",
    "response = requests.post(api, json=data, headers=headers)\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The predictions for the four instances were: 5.33, 6.09, 2.50, and 5.86 pounds respectively when I ran it (your results might be different)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### gcloud shell API\n",
    "\n",
    "Instead we could use the gcloud shell API. Create a newline delimited JSON file with one instance per line and submit using gcloud."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "Writing inputs.json\n"
      ]
     }
   ],
   "source": [
    "%%writefile inputs.json\n",
    "{\"is_male\": \"True\", \"mother_age\": 26.0, \"plurality\": \"Single(1)\", \"gestation_weeks\": 39}\n",
    "{\"is_male\": \"False\", \"mother_age\": 26.0, \"plurality\": \"Single(1)\", \"gestation_weeks\": 39}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now call `gcloud ai-platform predict` using the JSON we just created and point to our deployed `model` and `version`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "WEIGHT\n",
       "[6.434544563293457]\n",
       "[6.186980247497559]\n"
      ]
     },
     {
      "name": "stderr",
      "output_type": "stream",
      "text": [
       "WARNING: Using endpoint [https://ml.googleapis.com/]\n"
      ]
     }
   ],
   "source": [
    "%%bash\n",
    "gcloud ai-platform predict \\\n",
    "    --model=babyweight \\\n",
    "    --json-instances=inputs.json \\\n",
    "    --version=# TODO 3b: Add model version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab Task #4: Use model to make batch prediction.\n",
    "\n",
    "Batch prediction is commonly used when you have thousands to millions of predictions. It will create an actual Cloud AI Platform job for prediction. Complete __#TODO__s so we can call our deployed model on Cloud AI Platform for batch prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "jobId: babypred_200429_142714\n",
       "state: QUEUED\n"
      ]
     },
     {
      "name": "stderr",
      "output_type": "stream",
      "text": [
       "Copying file://inputs.json [Content-Type=application/json]...\n",
       "/ [1 files][  179.0 B/  179.0 B]                                                \n",
       "Operation completed over 1 objects/179.0 B.                                      \n",
       "CommandException: 1 files/objects could not be removed.\n",
       "Job [babypred_200429_142714] submitted successfully.\n",
       "Your job is still active. You may view the status of your job with the command\n",
       "\n",
       "  $ gcloud ai-platform jobs describe babypred_200429_142714\n",
       "\n",
       "or continue streaming the logs with the command\n",
       "\n",
       "  $ gcloud ai-platform jobs stream-logs babypred_200429_142714\n"
      ]
     }
   ],
   "source": [
    "%%bash\n",
    "INPUT=gs://${BUCKET}/babyweight/batchpred/inputs.json\n",
    "OUTPUT=gs://${BUCKET}/babyweight/batchpred/outputs\n",
    "gcloud storage cp inputs.json $INPUT\n",
    "gcloud storage rm --recursive --continue-on-error $OUTPUT \n",
    "gcloud ai-platform jobs submit prediction babypred_$(date -u +%y%m%d_%H%M%S) \\\n",
    "    --data-format=TEXT \\\n",
    "    --region ${REGION} \\\n",
    "    --input-paths=$INPUT \\\n",
    "    --output-path=$OUTPUT \\\n",
    "    --model=babyweight \\\n",
    "    --version=# TODO 4: Add model version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab Summary:\n",
    "In this lab, we set up the environment, deployed a trained Keras model to Cloud AI Platform, online predicted from deployed model on Cloud AI Platform, and batch predicted from deployed model on Cloud AI Platform."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright 2019 Google Inc. Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}